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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 29 July 2009 . 
2a )□ This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-9,19 and 21-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-9, 19 and 21-25 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1. This Second Non-Final action is responsive to the amendment fded on 7 129 2009. 

2. NOTE: The Examiner contacted applicants representative on 12 4 2009 lor a proposed examiners amendment, ho\ve\ or 
at the time applicant could not be reached. 

3 . In the amendment claims 1-9, 19 and 2 1 -25 are pending. Claims 22-25 are new. Claims 1,19 and 22 are the 
Independent claims. 



Withdrawn Rejections 

4. The 35 U.S. I'. 102(b) rejections of claims 1-14 with cited reference of Vegas (NI'L) has been withdrawn in light of the 
persuasive arguments. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this 
Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill 
in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

6. Claims 1-9, 19 and 21-25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Vegas (Vegas 2.0 Users 
Manual, 2000, Sonic Foundry, pgs 1-411 (In PDF Format)) in view of Gupta (U.S. 7,11 1,009, filed on Sep. 15, 1999). 

Regarding Independent claims 1 and 19, An apparatus for managing multimedia content, said apparatus comprising: 
a processor; an arrangement lor siippK ing multimedia content; an input interface lor permitting the selection, for 
observation, of at least one of the following modes associated with the multimedia content: an audio portion that 
includes video; and a video portion that includes audio; an arrangement for labeling observations of a selected mode, 
wherein said arrangement for labeling observations of a selected mode comprises: an arrangement for assigning 
semantic, multimedia content-based labels to segments of said observations of a selected mode; wherein said 
arrang ement lor as signing semantic , multimedia content-based labels is configured to: pro\ ide a label from a 
p redefined s et of muhimcd ia c ontent dcM:rip iors; and assign a ne w label no i prese nt in said prcd cilii cd -,ci of 
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arrangement; cmd an arrangement lor .storing said semantic multimedia content-based labels with the multimedia 
conlcnl. 

Vegas leaches a video editing software that includes an arrangement for supplying multimedia content such as the 
videos in the timeline show n in the figure (see page 29 in PDF format). The user selects the video or the audio for 
editing in the interlace furthermore allowing the observation of each as shown along the timeline. The track view holds 
both video and audio annotations thus including obsenations of an audio portion that includes video or a video portion 
that includes audio. Furthermore the editing software allows the user to arrange the video mode or the audio mode for 
labeling along the timeline (see pages 45-107 & 109-1 19). Vegas further shows on page 88 the use of a comments 
field to add labels lor noting significant obsen ations in a keyword textbox. The labeling include selecting a semantic 
label such as the multimedia segments w ith names associated w ith the content show n on pg 86 under list view which 
are in a predefined sel of multimedia descriptors and used to label by adding the clip/content or audio data to the media 
on the timeline. Vegas fails to explicitly teach the use of checkboxes for labeling foreground and background sounds. 
However Gupta suggests the use of checkboxes for annotating video and audio data (see column 8, line 54-67 & 
column 11, lines 64-67 to column 9 lines 1-67, regarding checkboxes see column 14, lines 5-67). At the time of the 
invention it would have been obvious for one of ordinary skill to have implemented the checkbox annotation system 
w ith Vegas's \ ideo editing softw are. The motivation for doing so would be to provide metadata annotation information 
for searching audio and \ ideo data saving the user time for determining relevant video/audio segments. 

Regarding Dependent claim 2, with dependency of claim 1, Vegas discloses wherein said input interface permits the 
selection, for observation, of both of the following associated with the multimedia content: an audio portion that 
includes video; and a video portion that includes audio (pages 29 & 45-107, including the explanation provided in the 
Independent claim). 

Regarding Dependent claim 3, with dependency of claim 1 , Vegas discloses wherein said input interface additionally 
permits the selection, for observation, of solely a video portion of multimedia content (pages 29 & 45-107, including 
the explanation provided in the Independent claim). 
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Regarding Dependent claim 4, with dependene} of claim I , Vegas discloses wherein said input interlace additionally 
permits the selection, for observation, of solely an audio portion of multimedia content ( pages 29 & 45-107, including 
the explanation provided in the independent claim). 

Regarding Dependent claim 5, with dependency of claim 1, Vegas discloses \\ herein said arrangement for supplying 
multimedia content comprises a working memory which stores amiotated multimedia files in an industry standard 
format (pages 29 & 45-107, including the explanation provided in the Independent claim). 

Regarding Dependent claim 6, with dependency of claim 1, Vegas discloses wherein said input interface is adapted 
to: first permit the selection of a multimedia file and then permit the selection of said at least one of: an audio portion 
simultaneously with video; and a video portion simultaneously with audio (pages 29 & 45-107, including the 
explanation provided in the Independent claim). 

Regarding Dependent claim 7, with dependency of claim 1, Vegas discloses a working memory for saving the 
annotated observations of a selected mode (pages 29 & 45-107, including the explanation provided in the Independent 
claim). 

Regarding Dependent claim 8, with dependency of claim 1, Vegas discloses wherein said input interlace is adapted to 
permit the selection, for obsen alion, at least the following mode associated with the multimedia content: a video 
portion that includes audio (pages 29 & 45-107, including the explanation provided in the Independent claim). 

Regarding Dependent claim 9, with dependency of claim 1, Vegas discloses wherein said input interface comprises: 
an arrangement for permitting the selection, for observation, of a video mode of multimedia content; and an 
arrangement for selectably adding audio to the video mode for observation (pages 29 & 45-107, including the 
explanation provided in the Independent claim). 

Regarding Dependent claim 21, with dependency of claim 1, Vegas tails to explicith teach the use o f checkboxes for 
labeling foreground and background sounds. However Gupta suggests the use of checkboxes for annotating video and 
audio data further disclosing wherein the arrangement for annotating observations of a selected mode is further 
configured to allow a user to check the check boxes for annotating foreground and background sounds hi the audio 
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being annotated and is further configured to permit the user to note observations not accounted for in the checkboxes 
for annotating foreground and background sounds in a keywords text box (see column 8, line 54-67 & column 1 1, lines 
64-67 to column 9 lines 1-67, regarding checkboxes see column 14, lines 5-67). At the time of the invention it would 
have been obvious for one of ordinary skill to ha\e implemented the checkbox annotation system with Vegas' s video 
editing software. The motivation for doing so would be to provide metadata annotation information for searching audio 
and video data saving the user time for determining relevant video/audio segments. 

Regarding Independent claim 22, A program storage de\ ice readable b> machine, tangibh embod\ ing a program of 
instructions executable by the machine to: permit selection, obsen ation and labeling of multimedia content in one or 
more modes; provide a plurality of check boxes in a user interface for labeling (he multimedia content with a 
predefined set of multimedia labels; and pro\ ide a ke\ word text box lor labeling the multimedia content. 

Vegas teaches a video editing software that includes an arrangement for supplying multimedia content such as the 
videos in the timeline shown in the figure (see page 29 in PDF format). The user selects the video or the audio for 
editing in the interface furthermore allowing the observation of each as shown along the timeline. The track view holds 
both video and audio annotations thus including observations of an audio portion that includes video or a video portion 
that includes audio. Furthermore the editing software allows the user to arrange the video mode or the audio mode for 
labeling along the timeline (see pages 45-107 & 109-1 19). Vegas further shows on page 88 the use of a comments 
field to add labels for noting significant obsen ations in a keyword textbox. The labeling include selecting a semantic 
label such as the multimedia segments with names associated with the content shown on pg 86 under list view which 
are in a predefined set of multimedia descriptors and used to label by adding the clip/content or audio data to the media 
on the timeline. Vegas fails to explicitly teach the use of checkboxes for labeling foreground and background sounds. 
However (itipla suggests the use of checkboxes for annotating \ ideo and audio data (see column 8, line 54-67 & 
column 11, lines 64-67 to column 9 lines 1-67, regarding checkboxes see column 14, lines 5-67). At the time of the 
invention it would have been obvious for one of ordinary skill to have implemented the checkbox annotation system 
with Vegas's video editing software. The motivation fordoing so would be to provide metadata annotation information 
for searching audio and video data saving the user time for determining relevant v ideo audio segments. 

Regarding Dependent claim 23, with dependency of claim 22, Vegas discloses wherein the one or more modes 
comprise a mode for labeling an audio portion of the multimedia content with or without associated video; and a mode 
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for labeling a video portion of the multimedia content with or without associated audio (pages 29 & 45-107, including 
the explanation provided in the Independent claim). 

Regarding Dependent claim 24, with dependency of claim 22, Vegas discloses wherein the program of instructions is 
further executable by the machine to: store labeled multimedia content as industn standard format files (pages 29 & 
45-107, including the explanation provided in the Independent claim). 

Regarding Dependent claim 25, with dependency of claim 22, Vegas discloses wherein the program of instructions is 
further executable by the machine to: store labels as MPEG-7 XML files (pages 29 & 45-107, including the explanation 
provided in the Independent claim). 



It is noted that any citation [[s]] to specific, pages, columns, lines, or figures in the prior art references and any 
interpretation of the references should not be considered to be limiting in any way. A reference is relevant for 
all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary 
skill in the art. [[See, MPEP 2123]] 



Response to Arguments 

7. Applicant's arguments filed 7/29/2009 have been fully considered but are moot in view of the new grounds of 

rejection. 



Conclusion 
References Cited 

The art made of record and not relied upon is considered pertinent to applicant's disclosure. 

• Gupta et al. (U.S. 6,956,593) discloses "User Interface For Creating, Viewing And Temporally Positioning 
Annotations For Media Content" 

• Gupta et al. (U.S. 6,484,156) discloses "Accessing Amiotations Across Multiple Target Media Streams" 
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Any inquiry concerning this communication or earlier communications from the examiner should be directed to 
Monglcsh M. Palcl whose telephone number is (571) 272-5937. The examiner can normally be reached M-W & F from 6 am- 
1 :30 pm & TH from 6 am -4 pm . 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Stephen S. Hong can be 
reached on (571) 272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application ma; be obtained from the Patent Application Information Retrieval 

(PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is a\ailablc through Private PAIR only. For more information about the 

PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the 

Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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